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Probability distributions having power-law tails are observed in a broad range of social, economic, 
and biological systems. We describe here a potentially useful common framework. We derive 
distribution functions {pk} for situations in which a ‘joiner particle’ k pays some form of price to 
enter a ‘community’ of size k — 1, where costs are subject to economies-of-scale (EOS). Maximizing 
the Boltzmann-Gibbs-Shannon entropy subject to this energy-like constraint predicts a distribution 
having a power-law tail; it reduces to the Boltzmann distribution in the absence of EOS. We show 
that the predicted function gives excellent fits to 13 different distribution functions, ranging from 
friendship links in social networks, to protein-protein interactions, to the severity of terrorist attacks. 
This approach may give useful insights into when to expect power-law distributions in the natural 
and social sciences. 


Probability distributions are often observed to have 
power-law tails, particularly in social, economic, and 
biological systems. Examples include distributions of 
fluctuations in financial markets [1], the populations of 
cities [5], the distribution of website links [3], and oth¬ 
ers mi]. Such distributions have generated much pop¬ 
ular interest [iiz] because of their association with rare 
but consequential events, such as stock market bubbles 
and crashes. 

If sufficient data is available, finding the mathemati¬ 
cal shape of a distribution function can be as simple as 
curve-fitting, with a follow-up determination of the sig- 
nihcance of the mathematical form used to fit it. On the 
other hand, it is often interesting to know if the shape of a 
given distribution function can be explained by an under¬ 
lying generative principle. Principles underlying power- 
law distributions have been sought in various types of 
models. For example, the power-law distributions of node 
connectivities in social networks have been derived from 
dynamical network evolution models [HHIT]. A large and 
popular class of such models is based on the ‘preferen¬ 
tial attachment’ rule [IHHIII, wherein it is assumed that 
new nodes attach preferentially to the largest of the ex¬ 
isting nodes. Explanations for power-laws are also given 
by Ising models in critical phenomena [281 - I34] . network 
models with thresholded ‘fitness’ values |3S] and random- 
energy models of hydrophobic contacts in protein inter¬ 
action networks [36]. 

However, such approaches are often based on partic¬ 
ular mechanisms or processes. They often predict par¬ 
ticular power-law exponents, for example. Our inter¬ 
est here is in finding a broader vantage point, as well 
as a common language, for describing a range of dis¬ 
tributions, from power-law to exponential. For deriv¬ 
ing exponential distributions, a well-known general prin¬ 
ciple is the method of Maximum Entropy (Max Ent) 
in statistical physics m l38] . In such problems, you 
want to choose the best possible distribution from all 
candidate distributions that are consistent with certain 


set of constrained moments, such as the average energy. 
For this type of problem, which is highly underdeter¬ 
mined, a principle is needed for selecting a ‘best’ math¬ 
ematical function from among alternative model distri¬ 
bution functions. To find the mathematical form of 
the distribution function pk over states k = 1,2,3,..., 
the Max Ent principle asserts that you should maximize 
the Boltzmann-Gibbs-Shannon (BGS) entropy functional 
S'[{pfc}] = — X^fePfclogpfc subject to constraints, such as 
the known value of the average energy {E). This pro¬ 
cedure gives the exponential (Boltzmann) distribution, 
Pk oc where /? is the Lagrange multiplier that en¬ 

forces the constraint. This variational principle has been 
the subject of various historical justifications. It is now 
commonly understood as the approach that chooses the 
least-biased model that is consistent with the known con¬ 
straint (s) |5U] . 

Is there an equally compelling principle that would se¬ 
lect fat-tailed distributions, given limited information? 
There is a large literature that explores this. Inferring 
non-exponential distributions can be done by maximiz¬ 
ing a different mathematical form of entropy, rather than 
the Boltzmann-Gibbs-Shannon form. Examples of these 
non-traditional entropies include those of Tsallis [40j . 
Renyi m, and others 1421 143j . For example, the Tsallis 
entropy is defined as i'l2kPk ~ AT is a con¬ 

stant and g is a parameter for the problem at hand. Such 
methods otherwise follow the same strategy as above: 
maximizing the chosen form of entropy subject to an ex¬ 
tensive energy constraint gives non-exponential distribu¬ 
tions. The Tsallis entropy has been applied widely |44|- 
[S3|. 

However, we adopt an alternative way to infer non¬ 
exponential distributions. To contrast our approach, we 
first switch from probabilities to their logarithms. Log¬ 
arithms of probabilities can be parsed into energy-like 
and entropy-like components, as is standard in statisti¬ 
cal physics. Said differently, a nonexponential distribu¬ 
tion that is derived from a Max Ent principle requires 
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FIG. 1. Hk is the joining cost for a particle to join a size k—1 community. This diagram can describe particles forming colloidal 
clusters, or social processes such as people joining cities, citations added to papers, or link creation in a social network. 


that there be non-extensivity in either an energy-like 
or entropy-like term; that is, it is non-additive over in¬ 
dependent subsystems, not scaling linearly with system 
size. Tsallis and others have chosen to assign the non- 
extensivity to an entropy term, and retain extensivity 
in an energy term. Here, instead, we keep the canon¬ 
ical BGS form of entropy, and invoke a non-extensive 
energy-like term. In our view, only the latter approach 
is consistent with the principles elucidated by Shore and 
Johnson (SJ) [37] (reviewed in [331). Shore and Johnson 
(SJ) showed that the Boltzmann-Gibbs-Shannon form 
of entropy is uniquely the mathematical function that 
ensures satisfaction of the addition and multiplication 
rules of probability. SJ asserts that any form of entropy 
other than BGS will impart a bias that is unwarranted 
by the data it aims to fit. We regard the SJ argument as 
a compelling first-principles basis for defining a proper 
variational principle for modeling distribution functions. 
Here, we describe a variational approach based on the 
BGS entropy function, and we seek an explanation for 
power-law distributions in the form of an energy-like 
function instead. 


I. THEORY 


particles is the sum. 


k-l 

Wk = 

i=i 


( 1 ) 


Max Ent asserts that we should choose the probability 
distribution that has the maximum entropy amongst all 
candidate distributions that are consistent with the mean 
value {w) of the total cost of assembly [33] . 
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where A is a Lagrange multiplier that enforces the con¬ 
straint. 

In situations where the cost of joining does not de¬ 
pend on the size of the community a particle joins, then 
where is a constant. The cumulative cost of 
assembling the cluster is then 

Wfc = (fc-l)/i°. (3) 

Substituting into Eq. [^and absorbing the Lagrange mul¬ 
tiplier A into yields the grand canonical exponential 
distribution, well-known for this problems such as this: 


A. Assembly of simple colloidal particles 

We frame our discussion in terms of a ‘joiner particle’ 
that enters a cluster or community of particles, as shown 
in Fig. [2 On the one hand, this is a natural way to de¬ 
scribe the classical problem of the colloidal clustering of 
physical particles; it is readily shown (reviewed below) to 
give an exponential distribution of cluster sizes. On the 
other hand, this general description also pertains more 
broadly, such as when people populate cities, links are 
added to websites, or when papers accumulate citations. 
We want to compute the distribution, pk , of populations 
of communities having size k = 1,2,...,A. 

To begin, we express a cumulative ‘cost’ of joining. For 
particles in colloids, this cost is expressed as a chemical 
potential, i.e., a free energy per particle. If pj represents 
the cost of adding particle j to a cluster of size j — 1, 
the cumulative cost of assembling a whole cluster of k 




i 

In short, when the joining cost of a particle entry is 
independent of the size of the community it enters, the 
community size distribution is exponential. 

B. Communal assemblies and ‘economies of scale’ 

Now, we develop a general model of communal assem¬ 
bly based on ‘economies of scale’. Gonsider a situation 
where the joining cost for a particle depends on the size of 
the community it joins. In particular, consider situations 
in which the costs are lower for joining a larger commu¬ 
nity. Said differently, the ‘cost-minus-benefit’ function 
Pk is now allowed to be subject to ‘economies of scale’, 
which, as we note below, can also be interpreted instead 
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as a form of discount in which the community pays down 
some of the joining costs for the joiner particle. 

To see the idea of economy-of-scale cost function, imag¬ 
ine building a network of telephones. In this case, a com¬ 
munity of size 1 is a single unconnected phone. A com¬ 
munity of size 2 is two connected phones, etc. Consider 
the first phone: The cost of creating the first phone is 
high because it requires initial investment in the phone 
assembly plant. And the benefit is low, because there is 
no value in having a single phone. Now, for the second 
phone, the cost-minus-benefit is lower. The cost of pro¬ 
ducing the second phone is lower than the first since the 
production plant already exists. And the benefit is higher 
because two connected phones are more useful than one 
unconnected phone. For the third phone, the cost-minus- 
benefit is even lower than for the second because the pro¬ 
duction cost is even lower (economy of scale) and because 
the benefits increase with the number of phones in the 
network. 

To illustrate, suppose the cost-minus-benefit for the 
first phone is 150, for the second phone is 80, and for the 
third phone is 50. To express these cost relationships, 
we define an ‘intrinsic cost’ for the first phone (joiner 
particle), 150 in this example. And, we define the differ¬ 
ence in cost-minus-benefit between the first and second 
phones as the discount provided ‘by the first phone’ when 
the second phone ‘joins the community’ of two phones. 
In this example, the first phone provides a discount of 
70 when the second phone joins. Similarly, the total dis¬ 
count provided by the two-phone community is 100 when 
the third phone joins the community. 

In this language, the existing ‘community’ is ‘paying 
down’ some fraction of the joining costs for the next par¬ 
ticle. Mathematically, this communal cost-minus-benefit 
function can be expressed as 




kf^ 

ko 


( 5 ) 


The quantity /ifc on the left side of Eq. [^is the total cost- 
minus-benefit when a particle joins a fc-mer community. 
The joining cost has two components, expressed on the 
right side: each joining event has an intrinsic cost fi° that 
must be paid, and each joining event involves some dis¬ 
count that is provided by the community. Because there 
are k members of the existing community, the quantity 
fik/ko is the discount given to a joiner by each existing 
community particle, where ko is a problem-specific pa¬ 
rameter that characterizes how much of the joining cost 
burden is shouldered by each member of the community. 
In the phone example, we assumed ko = 1. The value of 
fco = 1 represents fully equal cost sharing between joiner 
and community member: each communal particle gives 
the joining particle a discount equal to what the joiner 
itself pays. The opposite extreme limit is represented by 
fco —^ oo: in this case, the community gives no discount 
at all to the joining particle. 

The idea of communal sharing of cost-minus-benefit is 
applicable to various domains. It can express that one 


person is more likely to join a well-populated group on a 
social-networking site because the many existing links to 
it make it is easier to find (i.e., lower cost) and because 
its bigger hub offers the newcomer more relationships to 
other people (i.e., greater benefit). Or, it can express 
that people prefer larger cities to smaller ones because 
of the greater benefits that accrue to the joiner in terms 
of jobs, services and entertainment. (In our terminology, 
a larger community ‘pays down’ more of the cost-minus- 
benefit for the next immigrant to join.) We use the terms 
‘economy-of-scale’ (EOS), or ‘communal’, to refer to any 
system that can be described by a cost function such 
as Eq. in which the community can be regarded as 
sharing in the joining costs, although other functional 
forms might also be of value for expressing EOS. 

Rearranging Eq.[^ gives fXk = fJ-°ko/{k + ko) ■ The total 
cost-minus-benefit, Wk, of assembling a community of size 
k is 

k—1 ^ 

Wfc =/r°A:o^ ——^ = Ai°fcoV'(fc + fco) - C', (6) 

J + ko 


where tpik) = —7 -I- J ^ digamma function 

(7 = 0.5772... is Euler’s constant), and the constant term 
C = fi°koip{ko) + will be absorbed into the normal¬ 
ization. 

From this cost-minus-benefit expression (Eq. [^, for a 
given fcg, we can now uniquely determine the probabil¬ 
ity distribution by maximizing the entropy. Substituting 
Eq.§ into Eq. yields 


Pk 


koip(k+ko) 
Q- koi’ii+ko) 


( 7 ) 


Eq. describes a broad class of distributions. These 
distributions have a power-law tail for large k, with ex¬ 
ponent p°ko, and a cross-over at k = ko from expo¬ 
nential to power-law. To see this, expand ip(k + ko) 
asymptotically and drop terms of order l/k"^. This yields 
Wk ~ pL° ko^TL {k -\-ko — , so Eq. obeys a power-law 

Pk ^ {k + ko — \) ^ for large k. pk becomes a sim¬ 
ple exponential in the limit of /cg —> 00 (zero cost shar¬ 
ing). One quantitative measure of a distribution’s posi¬ 
tion along the continuum from exponential-to-power-law 
is the value of its scaling exponent, /r°/cg. A small expo¬ 
nent indicates that the system has extensive social shar¬ 
ing, thus power-law behavior. As the exponent becomes 
large, the distribution approaches an exponential func¬ 
tion. Eq. has a power-law scaling only when the cost 
of joining a community has a linear dependence on the 
community size. The linear dependence arises because 
the joiner particle interacts identically with all other par¬ 
ticles in the community. 

What is the role of detailed balance in our modeling? 
Eigure shows no reverse arrows from fc to fc — 1. The 
principle of maximum entropy can be regarded as a gen¬ 
eral way to infer distribution functions from limited infor¬ 
mation, irrespective of whether there is an underlying a 
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FIG. 2. Eq.[^ gives good fits {P > 0.05; see SI for details) to 13 empirical distributions, with the values of and fco given 
in Table 1. Points are empirical data, and lines represent best-fit distributions. The probability pk of exactly k is shown in 
blue, and the probability of at least k (the complementary cumulative distribution, is shown in red. Descriptions and 

references for these datasets can be found in the SI. 
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kinetic model. So, it poses no problem that some of our 
distributions, such as scientific citations, are not taken 
from ‘reversible’ processes. 


II. RESULTS 

Eq.0 and Fig. show the central results of this pa¬ 
per. Consider three types of plots. On the one hand, 
exponential functions can be seen in data by plotting 
logpk vs k. Or, power-law functions are seen by plotting 
logpk vs logfc. Here, we find that plotting logp^ vs a 
digamma function provides a universal fit to several dis¬ 
parate experimental data sets over their full distributions 
(Fig.§. Fig. I shows fits of Fq. to 13 datasets, using 
fi° and /cq 3-8 fitting parameters that are determined by a 
maximum-likelihood procedure. (See SI for dataset and 
goodness-of-fit test details.) fi° and ko characterize the 
intrinsic cost of joining any cluster, and the communal 
contribution to sharing that cost, respectively. 

Rare events are less rare under fat-tailed distributions 
than under exponential distributions. For dynamical sys¬ 
tems, the risk of such events can be quantified by the co- 
ejjjcient of variation (CV), defined as the ratio of the 
standard deviation (t^ to the mean {k). For equilib¬ 
rium/steady state systems, the CV quantifies the spread 
of a probability distribution, and is determined by the 
power-law exponent, p°kQ. Systems with small scaling 
exponents (/i°fco < 3) experience an unbounded, power- 
law growth of their CV as the system size N becomes 
large, ak/{k) ^ . This growth is particularly rapid in 

systems with 1.8 < < 2.2, because the average com¬ 

munity size {k) diverges at /r°fco = 2. For these systems, 
/3 = 1/2 is observed. Several of our datasets fall into this 
‘high-risk’ category, such as the number of deaths due to 
terrorist attacks (Table 1). 


TABLE I. Fitting parameters and statistics. 


Data set 

t>° 

fco 

(fc) 

N 

pko 

P 

GitHub 

9(1) 

0.21(2) 

3.642 

120,866 

2(2) 

0.78 

Wikipedia 

1.5(1) 

1.3(1) 

25.418 

21,607 

1.9(1) 

0.79 

PGP 

1(1) 

2.6(2) 

4.558 

10,680 

2.6(3) 

0.16 

Word adjacency 

3.6(4) 

0.6(1) 

5.243 

11,018 

2.1(3) 

0.09 

Terrorists 

2.1(2) 

1(1) 

4.346 

9,101 

2.2(3) 

0.38 

Facebook wall 

1.6(1) 

2.3(3) 

2.128 

10,082 

3.6(5) 

0.99 

Proteins, fly 

0.9(2) 

5(2) 

2.527 

878 

5(2) 

0.89 

Proteins, yeast 

0.9(1) 

4(1) 

3.404 

2,170 

3(1) 

0.48 

Proteins, human 

0.8(1) 

4(1) 

3.391 

3,165 

4(1) 

0.52 

Digg 

0.68(3) 

4.2(3) 

5.202 

16,844 

2.8(2) 

0.05 

Petster 

0.21(3) 

15(3) 

13.492 

1,858 

3(1) 

0.08 

Word use 

2.3(1) 

0.8(1) 

11.137 

18,855 

1.9(2) 

0.56 

Software 

0.8(1) 

2.1(3) 

62.82 

2,208 

1.7(3) 

0.69 


g 

s 
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FIG. 3. Eq. [^fitted to the 13 datasets in Table 1, plotted 
against the total cost to assemble a size k community, p°Wk-i. 
Values of /r° and fco are shown in Table 1. The j/-axis has 
been re-scaled by dividing by the maximum pk, so that all 
curves begin at pk/ma.x{pk) = 1. All data sets are fit by the 
logy = —X line. See Fig.for fits to individual datasets. 

III. DISCUSSION 

We have expressed a range of probability distribu¬ 
tions in terms of a generalized energy-like cost func¬ 
tion. In particular, we have considered types of costs 
that can be subject to economies of scale, which we 
have also called ‘community discounts’. We maximize 
the Boltzmann-Gibbs-Shannon entropy, subject to such 
cost-minus-benefit functions. This procedure predicts 
probability distributions that are exponential functions 
of a digamma function. Such a distribution function 
has a power-law tail, but reduces to a Boltzmann dis¬ 
tribution in the absence of EOS. This function gives 
good fits to distributions ranging from scientific citations 
and patents, to protein-protein interactions, to friendship 
networks, and to weblinks and terrorist networks - over 
their full distributions, not just in their tails. 

Framed in this way, each new ‘joiner particle’ must pay 
an intrinsic buy-in cost to join a community, but that 
cost may be reduced by a communal discount (an econ¬ 
omy of scale).Here, we discuss a few points. First, both 
exponential and power-law distributions are ubiquitous. 
How can we rationalize this? One perspective is given 
by switching viewpoint from probabilities to their loga¬ 
rithms, which are commonly expressed in a language of 
dimensionless cost functions, such as energy/RT. There 
are many forms of energy (gravitational, magnetic, elec¬ 
trostatic, springs, interatomic interactions, etc). The 
ubiquity of the exponential distribution can be seen in 
terms of the diversity and interchangeability of energies. 

A broad swath of physics problems can be expressed 
in terms of the different types of energy and their ability 
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to combine, add or exchange with each other in various 
ways. Here, we indicate that non-exponential distribu¬ 
tions too, can be expressed in a language of costs, partic¬ 
ularly those that are shared and are subject to economies 
of scale. Second, where do we expect exponentials vs. 
power-laws? What sets Eq. apart from typical en¬ 
ergy functions in physical systems is that EOS costs are 
both independent of distance and long-ranged (the joiner 
particle interacts with all particles in given community). 
Consequently, when the system size becomes large, due 
to the absence of a correlation length-scale, the energy 
of the system does not increase linearly with system size 
giving rise to a non-extensive energy function. This view 
is consistent with the appearance of power-laws in crit¬ 
ical phenomena, where interactions are effectively long- 
ranged. 

Third, interestingly, the concept of cost-minus-benefit 
in Eq. can be further generalized, also leading to 
either Gaussian or stretched-exponential distributions. 
A Gaussian distribution results when the cost-minus- 
benefit function grows linearly with cluster size, fj,k fc; 
this would arise if the joiner particle were to pay a tax 
to each member of a community. This leads to a total 
cost of Wk ^ (see Eq. [^. These would be ‘hostile’ 
communities, leading to mostly very small communities 
and few large ones, since a Gaussian function drops off 
even faster with k than an exponential does. An example 
would be a Goulombic particle of charge q joining a com¬ 
munity of k other such charged particles, as in the Born 
model of ion hydration m- A stretched-exponential dis¬ 
tribution can arise if the joiner particle instead pays a 
tax to only a subset of the community. Eor example, in a 
charged sphere with strong shielding, if only the particles 
at the sphere’s surface interact with the joiner particle, 
then ~ and Wk ~ k^^^, leading to a stretched- 
exponential distribution. In these situations, EOS can 
affect the community-size distribution not only through 
cost sharing but also through the topology of interac¬ 
tions. 

Einally, we reiterate a matter of principle. On the 
one hand, non-exponential distributions could be derived 
by using a non-extensive entropy-like quantity, such as 
those of Tsallis, combined with an extensive energy-like 
quantity. Here, instead, our derivation is based on using 
the Boltzmann-Gibbs-Shannon entropy combined with a 
non-extensive energy-like quantity. We favor the latter 
because it is consistent with the foundational premises 
of Shore and Johnson m- In short, in the absence of 
energies or costs, the BGS entropy alone predicts a uni¬ 
form distribution; any other alternative would introduce 
bias and structure into pk that is not warranted by the 
data. Models based on non-extensive entropies, on the 
other hand, intrinsically prefer larger clusters, but with¬ 
out any basis to justify them. The present treatment 
invokes the same nature of randomness as when physical 
particles populate energy levels. The present work pro¬ 
vides a cost-like language for expressing various different 
types of probability distribution functions. 
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Appendix A: Datasets and fitting 

In this paper, we have two aims: (i) to use Max Ent to 
identify a general functional form for situations involving 
cost-minus-benefit constraints, and (ii) to fit data over a 
broad range of contexts. For (i). Max Ent predicts an 
exponential of a digamma function, provided that fco is 
known. It also shows how to compute a full distribu¬ 
tion if we are given a single quantity, (w). For (ii), our 
aim is just to do simple curve-fitting of data, given the 
mathematical form from (i). In this case, we have no mi¬ 
croscopic model for fcg, so we know neither fcg nor fj,°. In 
this case, our objective is not to find the full distribution 
from (w). Rather, for (ii), we are given the full distribu¬ 
tion function, and our objective is to find the best values 
of ko and fi° that fit it. 

Our fitting procedure is as follows. We use the maxi¬ 
mum likelihood estimation function mle function in Mat- 
lab, with probability distribution specified by Eq. Ta¬ 
ble 1^ shows estimated fj,° and fcg values, with 95% con¬ 
fidence intervals. We calculate goodness-of-fit P-values 
using the Monte Carlo simulation procedure (based on 
the Kolmogorov-Smirnov test) described in [5]. The 13 
datasets shown here are above the P = 0.05 statistical 
significance threshold proposed in . P-values shown in 
Table SI are based on 1000 simulations for each dataset. 

We fit Eq. [^to 13 data sets: 

• Project membership on the social cod¬ 
ing website GitHub (downloaded from 
konect.uni-koblenz.de) 

• Edits made by users of the English-language 
Wikipedia [60] 



1/ko 


FIG. SI. plotted against 1/ko for the 13 data sets listed 
in Table ||j Error bars are 95% confidence intervals. Linear 
regression is shown as a solid line, fj.° = 0.516^(7^ — 0.125 
{R^ = 0.991). Each point on this plot represents an empirical 
data set. 


• Interactions between users of the Pretty Good Pri¬ 
vacy (PGP) secure data transfer algorithm [57] 


• Words occurring immediately after one an¬ 
other in a Spanish book (downloaded from 
konect.uni-koblenz.de) 


• Deaths resulting from terrorist attacks from Febru¬ 
ary 1968 to June 2006 [59] 


• Wall posts by users to their walls on the social¬ 
networking website Facebook, from a 2009 crawl of 
New Orleans Facebook m 


• Pairwise, physical protein-protein interactions 
(PPI) of proteins detected in small-scale PPI net¬ 
work data, in yeast {Saccharomyces cerevisiae), 
fruit flies (Drosophila melanogaster), and humans 
[Homo sapiens) [5S] 


• Replies between users of the social news website 
Digg [56] 


• Friendships between users of the Petster social 
networking site Hamsterster (downloaded from 
konect.uni-koblenz.de) 


• Occurrences of unique words in the novel Moby 
Dick [4] 


• Class-class dependencies in the software li¬ 
braries JUNG and javax (downloaded from 
konect.uni-koblenz.de) 


An overlay of all fits and datasets is shown in Fig. [^ 
Individual parameters and fits are shown in Table [l] and 
Fig.|2] 



